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1. INTRODUCTION 
Conjugate gradient (CG) method is used in finding the minimum value for unconstrained 
optimization problem. This method can be expressed in general form such; 


min f(x) (1) 


xeR" 


where {:R” —>Ris a continuously differentiable nonlinear function. g(x) is denoted as a gradient of the 


function. Equation (1) can be solved by using several methods such Steepest Descent and Newton method 
but CG method is the most preferred due to its simplicity [1]. The nonlinear CG method generates a sequence 
of {x, \by using the recurrence; 


Xp = XE +ad,, k =0,1,2.... (2) 


where x, is the current iterative point and x9 eR” is set to be a starting point of the sequence. From (2), 


a, > Ois known as a step size and d, is the search direction defined by the rule: 
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é if k=0 
‘=| ee (3) 


8+ hd if k21 


The most common technique in inexact line search used widely is strong Wolfe-Powell line search, 
where a, > Osatisfies 


F(X +Q AK) Sf Xp + 54,21. 4; . |B(x, +k)’ < oli ds| (4) 


with O0<d<o<1 are both constants. Distinct choice of the parameter /; yields different numerical 


performance. Past study has shown at least six formulae for /, , which are given as follows; 





pits = 8k (Bi ~ Bis) (5) 
(84 - Se) A 
FR _ Sik (6) 
lel? 
pyre — 8k (8x ~ 8x) (7) 
lex 
T 
CD &k Sk 
= — SK OK (8) 
8-1 
T 
DY 88k (9) 
(g, aa Sal Aa 
BRM — Bi (8x — 8-1) (10) 


eel! 


From (5)-(10), gj, and g,_; are the abbreviations of g(x;,) and g(x,_;) which are the gradients of 
f(x) at points x, and x,_; respectively. |le| denotes the Euclidean norm of the vectors. From (5)-(10), the 
above corresponding methods are respectively called as HS [2], FR [3], PRP [4], CD [5], DY [6], and lastly 
RMIL denotes for Rivaie, Mustafa, Ismail and Leong [7]. Zoutendijk [8] has proved the FR method in (6) to 
be globally converged under exact line search on a general function [9]. After that, [10] extended the result 
under strong Wolfe-Powell line search. A new modification of a conjugate gradient method is presented but 
it did not prove the global convergence under inexact line search though it has possess global convergence 
properties under exact line search [7&11]. The strong Wolfe-Powell line search is considered due to the 
higher cost of exact line search [12]. 

In this paper, a new coefficient with more simple /, is proposed. Section two will elaborate the 
motivation of the coefficient together with the algorithm. Section three will discuss and prove the sufficient 
descend condition and global convergence properties. Finally, conclusion and recommendation for future 
study are wrapped up at section four. 


2. NEW CG COEFFICIENT 
The new CG coefficient introduced is known as Be ea Vike MR is motivated mainly from [7] where 


the denominator is retained as same as in (10). Whilst, the nominator in (10) is given as oh (gy gi) which 


is as same as used in (5) and (7). During expansion, the nominator becomes gh gy ip ey which implies 
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2 
|| (ey). Choosing the right nominator is important due to its role as a restart property in avoiding 


problems associated with jamming, [13-14]. Preventing any negative value, modifications has been made 
[15]; hence the new CG coefficient and the simplified version are as follows; 








lex? - [ef gx] 
BeMR — maxi 0, ; ay) 
4x 
Before proceeds with more details steps, po" needs to be simplified; 
0 for Igeal’ < leku1se| le ' 
SMR k+1 
Fesl= Neil ekg] fecal SFr $ (12) 


2 
lez 





2 ~ 2 
al lel 


Algorithm 2.1: Conjugate Gradient Method 
A complete algorithm of CG method could be generated as follows: 


Step 1: Initialization. Set k =0 and select %) ER", dy =—-gy. if go =O, stop. 

Step 2: Based on (11), compute poe ; 

Step 3: Compute search directions d; based on (3). If||g,||<  , then stop. Otherwise, go the next step. 

Step 4: Based on (4), solve for Q; . 

Step 5: Updating new initial point using (2). If f(%41) < f(%q) and ||¢,||<¢ then, stop. Otherwise go to 
Step 3 withk=k+1. 


3. THEORETICAL ANALYSIS 
This section discussed and analysed the sufficient descent property for the new coefficient under 
strong Wolfe-Powell line search direction. Before proceed, let assume that g, #0 for all k or else, the 


stationary point has been found. For any iterative method to be globally convergent, it is important to suffice 
its descent property, that is; 


gidp<0, 


3.1. Sufficient Descent Property 
Before proceed, let assume that g, #Ofor all k or else, the stationary point has been found. For any 


iterative method to be globally convergent, it is important to suffice its descent property, that is; 


ghd, <—clg,|/ (13) 


where Cis a positive constant, is crucial to ensure the global convergences of the nonlinear conjugate 
gradient method under strong Wolfe-Powell line search direction [16]. Sufficient descent property is 


important to show that the function f(x) can be reduces along the search direction. The proving steps below 
are modified from [11-12]. 


Theorem 3.1 


If g, and d, are generated by algorithm 2.1 witho < ~ , then, for all k =O, it becomes; 
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Isell < 2 (14) 
lal 3 

Proof 

lgoll _ 


[ao 


Suppose for some k = 0, (14) holds true. Rearrange (3) and multiplying it with Liat , then 


The proving steps are performed by inductions. For k =Qand,;— . =1 <2 =, hence (14) holds fork =0. 


Sinden =e.’ + Be Sid (15) 


From strong Wolfe-Powell condition and absolute values properties in (4), expression (15) becomes; 























geal st ghudia +|Pruskude| Igeull s gedit +o/Beailgh ae (16) 
Since BoM* > 0, then 

Igeull s lgtdes + Of gid;| (17) 
By using Cauchy inequalities and substituting (12) in (17), 

[seal <feallnaeo eas sell as 

dy 

Implies; 

leva Slevlbial-+oleal (19 
Applying the induction hypothesis in (14), 

Jeol? slecalldcsil+2ofecsl  [eeul?-20)sJecsulldces| Beth <3 20) 

3 3 ld.) + G-So) 


[seul 5 < 
east 





Therefore, if o < =. then a nienee; (14) is true fork +1. The proof is completed. 


3.2. Global Convergence Properties 

The following assumption is needed in order to proceed with the proof of global convergence 
properties. The proof modifications are from [11-12,17-19] 
Assumption 4.1 


1) f is bounded below on the level set R” and is continuous and differentiable in a neighborhood N of the 
level set = te ER"IfWS f(x) fat the initial point Xp. 
2) The gradient 9(x) is Lipschitz continuous in N, so there exists a constant L>Q such that; 


g(x) - g(y)|< L]x- yl] forany xy eN (21) 


From (11) and (13), 
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2. 
[sel if gid, <0 
5 < 
\as™*|< ld (22) 
Isl 
if gid, >0 
Kea ie 


Theorem 4.2 
Suppose that Assumption 4.1 holds. Consider any CG method in the form of (2) and (3) where a is 
obtained from (4). If the descend condition holds, then; 


lim infl}g,||=0 (23) 
0 


Proof 
To prove Theorem 4.2, contradiction method is used. That is, if Theorem 4.2 is not true, then there 


exists a constant ¢ >, such that; 

lsi|2¢ (24) 
Rewriting (3) as dys) + 8x41 = Ayp41d, and squaring both sides the equation; 

eel” = Algal — 28h 1a + (Best) lel” (25) 


From (22), if g, d, <0, then; 





4 
& 
desl = lee — 2g) dea t+ | kl (26) 


2 
[ae 
Divide (26) by lgcail" and from (13), 


desl” < 1 4 2c 











< (27) 
geal! geal seul 
Suppose that (23) does not hold, then there exists ¢ >Osuch that (24) holds for all k >0 
[desl gt 26 22651 IIc I’. gel 
<— + (28) 
Igeul erg gt [dics = “ea Ds Idea 
Also, from (22), if g/d, >0, then, 
|aecal|” ee | c [dieeall” gy tes eel 
47 zt 2 Zoe Pia: 
lexull a I llgeul” [seul e& & € 
seul’ , °F stalls (29) 





leeual oH C5 ldeal? 


From (28) and (29), this shows that (23) holds. The proof is completed. 
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4. CONCLUSION 
RMIL 


By taking a little modification at B, “in (10), a new coefficient pe 


of the conjugate gradient 


method has been proposed. Results showed that the new coefficient satisfy the sufficient descent conditions 
and converge globally under strong Wolfe-Powell line search. It is proved that the algorithm is practical and 
effective to be used. Following with the proving provided, the coefficient will be tested on certain test 
function for future study. 
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